OPEN 3 ACCESS Freely available online 



PLOS 



ONE 



The Role of Opportunistic Migration in Cyclic Games 



Pierre Buesser^, Marco Tomassini 

Faculty of Business and Economics, University of Lausanne, Lausanne, Switzerland 



CrossMark 



Abstract 

We study cyclic evolutionary games in a spatial diluted grid environment in which agents strategically interact locally but 
can also opportunistically move to other positions within a given migration radius. We find that opportunistic migration can 
inverse the cyclic prevalence between the strategies when the frequency of random imitation is large enough compared to 
the payoff-driven imitation. At the transition the average size of the patterns diverges and this threatens diversity of 
strategies. 
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Introduction 

Cyclic behavior can be observed in evolutionary games when 
there are more than two strategies available to the players, a well- 
known case being the Rock-Scissors-Paper (RSP) class of games [1]. 
This behavior is not only of theoretical interest since it is partly 
responsible for the biodiversity on Earth, and has been actually 
observed in several biological situations such as the dynamic 
behavior of side-blotched lizards populations [2], coral reef 
invertebrates [3], and competition among different bacteria 
strands [4] among others. These games have been studied 
extensively both theoretically and by computer simulations. 
Rigorous results are available for well mixed populations in the 
infinite population size limit pointing to the fact that the system 
may converge toward a stable or Lyapunov stable interior rest 
point, or to an unstable rest point leading to an heteroclinic cycle, 
depending on the relative values of the payoffs (see, for example, 
[1,5,6]). Cyclic behavior has also been found in studies of the 
public goods game type when players, besides being able to choose 
between cooperating or defecting behavior, also have the choice of 
not taking part in the game (so-called "loner" strategy) [7]. 
Interestingly, a little later this oscillating behavior was actually 
observed in an experiment with human subjects by D. Semman 
et al. [8]. Likewise, in a spatial setting such as two-dimensional 
grids or, more generally, on relational networks, several results 
have been obtained. Szabo and Hauert [9] and Szabo and Vukov 
[10] studied the Prisoner's Dilemma on two-dimensional grids 
with three strategies: cooperate, defect, and loners and observed 
that the three strategies survive in a cyclic dominance way akin to 
the RSP game. A similar phenomenon manifests itself on random 
graphs but with different characteristics. In [11] Szabo et al. 
investigated the behavior of the RSP game on regular small- world 
networks. In more recent work A. Szolnoki and coworkers have 
further studied the evolutionary Prisoner's Dilemma on spatial 
grids and random graphs showing that with a third tit-for-tat 



strategy the system can show a variety of interesting behaviors 
including stationary and oscillatory states [12]. When agents can 
only cooperate or defect but have time-dependent learning 
capabilities Szolnoki et al. [13] showed that cooperator and 
defectors can coexist and propagating waves appear in the 
spatially extended system. 

In another strand of research players also have the possibility of 
moving around in space, a feature that is central in ecosystems. 
Spatial travelling waves and cyclic dominance are typical features 
of these more biologically realistic settings which are often based 
on stochastic partial differential equations discretized on a grid to 
model random diffusion [14,15]. Another recent paper employs a 
continuous time space/time formalism in the RSP game with a 
non-diffusive spatial component [16]. The spatial flux is based on 
local gradients of relative fitness. In this respect, this study is closer 
to our approach described below but if focuses on pattern 
formation and dynamics. Indeed, the strategies are distributed at 
the start and remain fixed. While the system shows the formation 
of spirals in space for some initial conditions, and of strategy 
domains for others, since strategy proportions do not change 
extinction phenomena are absent. Other important recent works 
dealing with migration in diluted grid systems are [17,18]. 

In this paper we present a new model based on RSP games in 
which agents enjoy mobility but their displacements are not 
random; rather, they change place in a purposeful manner. 
Contingent mobility has previously been used under various forms 
in two-strategies evolutionary games of the Prisoner's Dilemma, 
Hawk-Dove, or Stag Hunt types [19-24]. The idea here is that the 
agents possess some basic reactive or elementary reasoning 
capability that allow them to sense the situation in their local 
spatial environment and to employ some simple heuristic to move 
accordingly. Heuristics range from very simple ones such as 
cooperators moving away from surrounding defectors when the 
latter are in the majority [19,23], to more elaborate ones such as 
"success-driven migration" where agents may try many destina- 
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Figure 1 . Average diversity levels with random migration (first row) and opportunistic migration (second row) as a function of the 
game radius Rg and the migration radius R^* The size of the grid is L = 50 and the density p is 0.5. In all cases the initial strategies of the players 
are attributed uniformly at random. Diversity is maximal for light tones and disappears for black tones as can be seen in the color code bar of Fig. 2. 
doi:1 0.1 371 /journal.pone.00981 90.g001 



tions in space and choose to jump to the most favorable one in 
terms of expected payoff [20,24]. Here agents use a simplified 
form of an heuristic introduced in [24] which consists in randomly 
trying one single free position in space within a given migration 
radius and to move there if it is empty and more profitable than 
the starting one. Our setting requires minimal rational capabilities 
on the part of the players but it is clearly not adequate for low-level 
biological organisms such as bacteria where it is likely that 
movements are almost random. On the other hand, the heuristics 
used are within the reach of many superior animal populations 
and certainly of humans. We show in the paper that the addition 
of opportunistic migration notably changes the dynamical 
behavior of species. In particular conditions, spatial traveling 
waves become much longer and tend to diverge with respect to the 
finite system size causing strategy extinction and thus threatening 
diversity. On the other hand, in different contexts this result could 
be seen as a positive one as it tends to stabilize an oscillating 
system. 

Methods 

We investigate a class of two-person, three-strategy, symmetric 
rock-scissors-paper game as a metaphor for cyclic behavior. These 
games have the generic payoff matrix M (equation 1) which refers 
to the payoffs of the row player. The payoff matrix for the column 
player is simply the transpose since the game is symmetric. 
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Figure 2. Diversity phase space in a well-mixed population as a 
function of the game's payoffs b\ and bj with a = 0. Diversity is 
maximal for light tones and disappears for black tones. 
doi:1 0.1 371/journal.pone.00981 90.g002 

Where b\ and are positive. The set of strategies is 

The Euclidean two-dimensional space is modeled by a discrete 
square lattice of side L with toroidal borders. Each vertex of the 
lattice can be occupied by one player or be empty. The density is p 
and N is the number of players. Players can interact with k 
neighbours which lie at an Euclidean distance smaller or equal 
than a given constant Rg. Players can also migrate to empty grid 
points at a distance smaller than R^. We use three neighborhood 
sizes with radius 1.5, 3, and 5; they contain, respectively, 8, 28, 
and 80 neighbours around the central player. 
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Figure 3. Average wavelength after time r = 5000 as a function of p with random migration and opportunistic migration. i^^ = 1.5, 
Rm = 1 .5,3,5. Left image : = 1 .5, b2 = 0.5 {game 1). Right image : bi = 0.5, ^2 = 1-5 {game 2). The size of the grid is L = 200 and the density p is 0.5. In 
all cases the initial strategies of the players are randomly attributed. 
doi:1 0.1 371 /journal.pone.00981 90.g003 



Each agent / interacts locally with a set of neighbours Vi lying 
closer than Rg. Let (Ti(t) be a vector giving the strategy profile at 
time t with 5'i =(1,0,0), ^2 = (0,1,0), and ^3 = (0,0,1), and let M 
be the p ay ofF matrix of the game (equation 1). The quantity 



n,(0=^a,(OMaJ(0 



(2) 



is the cumulated payoff collected by player / at time step t. 

We use the imitative strategy update called the Fermi rule [5] in 
which the focal player / is given the opportunity to imitate a 
randomly chosen neighbour j with probability: 



1 



l^expi-PiHj- nO) 



(3) 



where Hj — 11/ is the difference of the payoffs earned by j and / 
respectively and is a constant corresponding to the inverse 
temperature for the imitation update. When (high temper- 

ature) the probability of imitating y tends to a constant value 0.5 
and when f^^co (low temperature) the rule becomes deterministic: 
/ imitates j if (Hy — H^) > 0, otherwise it doesn't. In between these 
two extreme cases the probability of imitating neighbour j is an 
increasing function of Hy— 11/. 

We use an asynchronous Monte Carlo [5] scheme for strategy 
update and migration, i.e. players are updated one by one by 
choosing a random player in each step with uniform probability 
and with replacement. Then the player migrates with probability 
1/2, otherwise it updates its strategy. 

If the pseudo-random number drawn dictates that / should 
migrate, then the player considers a randomly chosen position in 
the disc of radius R^ around itself If the position is already 
occupied the player does not migrate, otherwise the player 
computes the payoff that it would obtain in that place with its 
current strategy. Then player / stays at its current position if it 
obtains higher payoff there, or migrates to the trial position in the 
opposite case. In order to introduce noise in the migration player / 
can decide to migrate with probability : 



/7(X^^X/) = 



(4) 



where is the difference of the payoffs earned by player / 

in the positions X/ and X/^ , where X/^ is the original position of 
player i and is a constant corresponding to the inverse 
temperature for the migration. We call these migrations opportu- 
nistic or fitness-based. 

We use two measures in order to assess diversity. The first one is 
called diversity and is simply the normalized product of the 
strategy frequencies : (/2i/22^3)/(l/3)^. It is proportional to the 
probability that three randomly chosen players adopt different 
strategies. Here the highest value of the product is reached when 
the distribution of the strategies is homogeneous, and if one or 
more strategy has vanished diversity becomes zero. Indeed, when 
there are only two strategies remaining, dominance will cause one 
of the two to disappear afterwards. 

The second measure is called the wavelength. It is a rough 
empirical approximation for the wavelength of a traveling wave or 
simply for the size of a domain where more than half of the players 
adopt locally the same strategy. We compute the width of a 
domain surrounding a player along the x axis dx and y axis dy and 
then choose the shortest width among dx and dy and take the 
average over all players p. Note that we could obtain similar results 
by taking the average over dx and dy. In order to obtain the 
wavelength around a player p with strategy s we compute the 
distance to the border of the s domain along the x and y axis in the 
positive and negative direction around the player p. In order to 
detect if a site / is inside a domain of players adopting strategy ^, 
we compute the frequency of players with that strategy inside the 
Moore neighborhood [Rg = 1.5) of/, including /. If the frequency is 
smaller than 0.5, / is considered to be out of the domain. 
Practically we move gradually on the axis until we reach the end of 
the domain. The next steps take into account the case where the 
spatial distribution of the population contains empty regions, i.e 
the frequencies of strategies cannot be computed. In that case, if 
there are no players in the neighborhood of /, the position of / is 
incremented. Then, if the new place is in a domain with the same 
strategy we consider that it is still the same domain and continue to 



PLOS ONE I www.plosone.org 



3 



June 2014 | Volume 9 | Issue 6 | e98190 



The Role of Opportunistic Migration in Cyclic Games 




! 1.0 0.03 0.001 

Figure 4. Screenshots with random migration (upper images) compared with opportunistic migration (lower images) as a function 

of ft Rg = 1.5, Rm = 1.5, bi = 1.5, and b2 = 0.5 (game 1). The size of the grid is L = 400 and the density p is 0.5. In all cases the initial strategies of the 
players are randomly attributed. Each color is associated with a different strategy: is yellow, ^2 corresponds to blue, and S3 is depicted in orange. 
doi:1 0.1 371 /journal.pone.00981 90.g004 



increment the test position. Otherwise, the position is considered 
to be out of the domain and the width of the region without 
players is subtracted from the total width. 

Next, we present here the measure for the invasion speed. We 
call this measure cyclicity and it takes values g[— 1,1]. The 
cyclicity measure for a player at a given time step / is 1 if the 
strategy has changed according to the natural cycling order 
{S\^S2^S3^S\) between t—\ and t, 0 if the strategy has not 
changed and — 1 if the strategy changed in the opposite way. The 
global cyclicity is the average of this quantity over the players 
during a time interval T after the system has evolved for t time 
steps. 



For the numerical simulations, the diversity phase-space 
generated by bi and /?2 has been sampled with a step of 0.1 and 
each value in the phase space reported in the figures is the average 
of /I = 50 independent runs. For the wavelength plots the number 
of independent runs is ^ = 200. The evolution proceeds by first 
initializing the population by adding players on grid cells with 
probability p. Then the players' strategies are initialized uniformly 
at random such that each strategy has a fraction of approximately 
1/3. We let the system evolve for a period of 1000 time steps 
for phase-space diagrams and ^ = 5000 for wavelength plots. In 
each time step N players are chosen for update. We then let the 
system evolve for t further steps and take the average measure 




Figure 5. Average cyclicity after time r = 5000 as a function of p with random and opportunistic migration. Rg = 1.5, Rm = 15, 3, 5. Left 
image :bi = l .5, ^2 = 0.5 {game 1). Right image : bi = 0.5, b2 = \.5 {game 2). The size of the grid is L = 200 and the density p is 0.5. In all cases the initial 
strategies of the players are randomly attributed. 
doi:1 0.1 371/journal.pone.00981 90.g005 
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Figure 6. Average wavelength for opportunistic migration after r = 5000 as a function of p for gamel: b\ = 1.5, ^2 = 0.5. L = 400 and 

i?„, = 1.5,3,5. In all cases the density p is 0.5 and the initial strategies of the players are randomly attributed. 
doi:1 0.1 371 /journal. pone.00981 90.g006 



value in this interval. Finally we report the average diversity or 
wavelength values over the n repetitions. 

Results 

In order to obtain an overview of the effect of opportunistic 
migration, the diversity measure is displayed as a function of the 
game parameters b\ and b2 for several values of j5. Fig. 1 depicts 
the diversity phase-space for a lattice of size L = 50 after time 
T = 1000 as a function of jS, Rg and Rm- The upper images refer to 
the random migration case, used here as a benchmark case, and 
the lower images refer to the opportunistic migration case. By 
comparing with the well-mixed case shown in Fig. 2, it can be 
observed that diversity can thrive in adverse games (lower left 
quadrant) when the interactions radius Rm and Rg are short 
{Rg,Rm<5). However this does not hold in the opportunistic 
migration case for all values of as can be seen in Fig. 1. For 
jS = 0.1 and 0.01 a small game radius Rg creates the opposite effect 
for i?^ = 1.5,3,5 : extinction extend in the upper right quadrant 
where diversity thrives in the ideal well-mixed case such that 
nearly all the games of the phase-space lose diversity. For higher 
game radius Rg = 5 the game space where full diversity thrives is 
similar to the one found in the random migration case. However 
this does not imply that the wavelength is similar in the extinction 
region. Although the small system size used for this exploratory 
analysis may cause finite-size effects i.e., extinction due to 
fluctuations, the results show that there is perhaps an interesting 
phenomenon occurring when ^ is tuned and thus we try to 
elucidate it further in the following. 

We study the wavelength on larger lattices as a function of fi 
since too small lattices do not let us appreciate large wavelengths 
due to finite size effects. Since the systematic study of the full game 
phase space would be computationally too heavy, we report the 
wavelength for two representative games in the plane. The first 



game {game 1) is in the middle of the left lower quadrant of the 
phase space, bi = 1.5; /?2 =0.5, and the second game [game!) is in 
the middle of the right upper quadrant, bi =0.5; b2 = 1.5. Fig. 3 
depicts the wavelength as a function of R^ and for game 1 and 
gamel, and Rg = 1.5, and a frequency of migration of 1/2. In the 
opportunistic migration case a marked peak appears for values of 
between 0.01 and 0.1. Results for a frequency of migration of 1/4 
and of 3/4 respectively are reported in Material S 1 . Fig. 4 displays 
some typical snapshots around the phase transition for random 
and opportunistic migration. In the central image of the lower row 
it is clearly visible how domains become larger and extinction sets 
in for jS = 0.03 with opportunistic migration. In Fig. 5 the average 
cyclicity is plotted as a function of for the opportunistic and 
random migration cases. It can be seen in the opportunistic 
migration case that the cyclicity vanishes at the peak and is slightly 
reversed on the left of the transition so that the position of the peak 
corresponds to the inversion of the cycling order. This effect can 
be explained in the extreme case where the imitation tends to 
be random but the migration is opportunistic. In that case, the 
players adopting a strategy St which is payoff-dominated by a 
strategy form clusters at the border between the two strategy 
regions since they try to minimize the number of players in 
their neighborhood. Meanwhile the players adopting the strategy 
are attracted toward the Si clusters and surround them with a 
smaller density. Since the strategy update rule is almost random 
imitation for very small fi the more clustered players spread their 
strategy faster than the surrounding players. In fact this effect can 
be understood in a bipartite population with two degree 
homogeneous sub-populations pi and p2 where players imitate 
randomly their neighbors. A quick calculation shows that the size 
of the sub-population which has the largest average degree spreads 
its strategy faster (see Material SI). Also in Material Slit is 
explained how the effect works using the example of a specific 
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spatial configuration consisting of two neighboring infinite regions 
with difFerent strategies. In the random migration case it is more 
difficult to find an explanation since there is no clustering, but the 
phenomenon is weaker and the peak is less marked. The increase 
of the wavelength when the cyclicity vanishes is not new and has 
been studied in [25] in a cyclic voter model with three strategies 
and a probability to imitate the dominant (dominated) strategy P 
respectively (1— P) but the phenomenon is not caused by 
migration, as in our case, since agents don't move and only the 
P parameter varies. In [26] authors study a spatial five-species 
predator-prey model with site exchange and invasions between 
neighbors according to the Rock-Paper-Scissors-Lizard-Spock 
game. They study the invasion velocities and species density 
fluctuations as a function of the invasion rates. It is reported that 
the fluctuations of species frequency diverge and invasion velocities 
between associations of strategies vanish when tuning the invasion 
rates. Coming back to the opportunistic migration case, we have 
checked that the inversion is stable with growing system size. Using 
short simulation times, such that the system has not reached 
extinction which means that this data is about the (initial) transient 
period of the system and not yet at the stationary state, cyclicity 
can be measured we show that the inversion is similar for all 
system sizes studied (see figures in Material SI). In Fig. 6 we 
display the average wavelength for L = 400, Rm = 1.5,3,5 and for 
game 1: Z?i = 1.5, b2 =0.5. By comparing with the corresponding 
curve in fig. 3 where L = 200 we remark that the peak becomes 
sharper for L = 400 thanks to the larger system size. This is due to 
the fact that the system can reach extinction before the end of the 
simulation due to fluctuations of the wavelength even if the mean 
wavelength is smaller than the system size. 

Finally, we study the effect of noise on the migration process 
using the Fermi rule with parameter (see Methods section). We 
observe that, as is decreased, the system undergoes a transition 
inside an interval where the phenomenon gradually disappears. 
(See Fig. S3 in Material SI). Thus, the global effect of migration 

References 

1. Hofbauer J, Sigmund K (1998) Evolutionary Games and Population Dynamics. 
Cambridge, N. Y. 

2. Sinervo B, Lively CM (1996) The rock-scissors-paper game and the evolution of 
alternative male strategies. Nature 380: 240-243. 

3. Jackson JBC, Buss L (1975) Allelopathy and spatial competition among coral reef 
invertebrates. Proc Natl Acad Sci 72: 5160-5163. 

4. Kerr B, Riley MA, Feldman MW, Bohannan BJM (2002) Local dispersal 
promotes biodiversity in a real-life game of rock-paper-scissors. Nature 418: 
171-174. 

5. Szabo G, Fath G (2007) Evolutionary games on graphs. Physics Reports 446: 
97-216. 

6. Sandholm WH (2010) Population Games and Evolutionary Dynamics. MIT 
Press, Cambridge, MA. 

7. Hauert C, DeMonte S, Hofbauer J, Sigmund K (2002) Volunteering as Red 
Queen mechanism for cooperation in public goods games. Science 296: 1129- 
1132. 

8. Semman D, Krambeck HJ, Milinski M (2003) Volunteering leads to rock-paper- 
scissors dynamics in a public goods game. Nature 425: 390 393. 

9. Szabo G, Hauert G (2002) Evolutionary prisoners's dilemma game with 
voluntary participation. Phys Rev E 66: 062903. 

10. Szabo G, Vukov J (2004) Cooperation for volunteering and partially random 
partnership. Phys Rev E 69: 036107. 

1 1 . Szabo G, Szolnoki A, Izsak R (2004) Rock-scissors-paper game on regular small- 
world networks. J Phys A: Math Gen 37: 2599-2609. 

12. Szolnoki A, Perc M, Szabo G (2010) Phase diagrams for three-strategy 
evolutionary prisoner's dilemma games on regular graphs. Phys Rev E 80: 
056104. 

13. Szolnoki A, Wang Z, Wang J, Zhu X (2010) Dynamically generated cyclic 
dominance in spatial prisoner's dilemma games. Phys Rev E 82: 036110. 



noise is to prevent extinction provided that it is high enough, i.e. 

less than 0.2. Of course, as migration noise increases, the 
situation resembles more and more to random walk migration, as 
it should. 

Discussion 

We studied the diversity of strategies in a RSP game in a spatial 
layout where players migrate opportunistically to more favorable 
places in their neighborhood. Differently from the many RSP-like 
systems that have been studied previously in which diffusion is 
either absent or is random, we found that the diversity is not 
maintained for large areas of the games' phase space, leading to 
strategy extinction, when the exponent of the strategy update rule 
is such that the imitative update is sufficiently noisy. Furthermore, 
studying the size of the patterns for two representative games as a 
function of fi we found that a transition occurs where the size of 
the patterns diverges and the prevalence of the strategies is 
reversed. Finally, we also introduced a migration noise and we 
found that if this noise is larger than a threshold the divergence of 
the wavelength disappears. 

Supporting information 

Material SI 

(PDF) 

Aclcnowledgments 

The authors thank A. Szolnoki for critically reading an early version of the 
manuscript and for his insightful comments. 

Autlior Contributions 

Conceived and designed the experiments: PB MT. Performed the 
experiments: PB. Analyzed the data: PB. Wrote the paper: MT PB. 



14. Reichenbach T, Mobilia M, Frey E (2007) Mobility promotes and jeopardizes 
biodiversity in rock-paper-scissors games. Nature 448: 1046-1049. 

15. MX, Wang WX, Lai YC, Grebogi C (2010) Cyclic competition of mobile 
species on continuous space: pattern formation and coexistence. Phys Rev E 82: 
066211. 

16. deForest R, Belmonte A (2013) Spatial pattern dynamics due to the fitness 
gradient ux in evolutionary games. Phys Rev E 87: 062138. 

17. Wang Z, Szolnoki A, Perc M (2012) If players are sparse social dilemmas are too: 
Importance of percolation for evolution of cooperation. Scientific Reports 2: 
369. 

18. Wang Z, Szolnoki A, Perc M (2012) Percolation threshold determines the 
optimal population density for public cooperation. Phys Rev E 85: 037101. 

19. Aktipis GA (2004) Know when to walk away: contingent movement and the 
evolution of cooperation. Journal of Theoretical Biology 231: 249 2160. 

20. Helbing D, Yu W (2009) The outbreak of cooperation among success-driven 
individuals under noisy conditions. Proc Nad Acad Sci USA 106: 3680-3685. 

21. Gong R, Wu B, Qiu Y, Wang L (2012) Evolution of cooperation driven by 
reputation-based migration. PLOS ONE 7: 35776. 

22. Ghen Z, Gao J, Kai Y, Xu X (2011) Evolution of cooperation among mobile 
agents. Physica A 390: 1615-1622. 

23. Jiang LL, Wang WX, Lai YC, Wang BH (2010) Role of adaptive migration in 
promoting cooperation in spatial games. Physical Review E 81: 036108. 

24. Buesser P, Tomassini M (2013) Opportunistic migration in spatial evolutionary 
games. Phys Rev E 88: 042806. 

25. Tainaka K, Itoh Y (1991) Topological phase transition in biological ecosystems. 
Europhys Lett 15: 399. 

26. Vukov J, Szolnoki A, Szabo G (2013) Diverging fluctuations in a spatial five- 
species cyclic dominance game. Phys Rev Lett 88: 022123. 



PLOS ONE I www.plosone.org 



6 



June 2014 | Volume 9 | Issue 6 | e98190 



